MAXSIM: An Automatic Metric for Machine Translation Evaluation Based on Maximum Similarity

This paper describes our participation in the NIST 2008 MetricsMATR Challenge, using our recently proposed automatic machine translation evaluation metric MAXSIM. The metric calculates a similarity score between a pair of English system-reference sentences by comparing information items such as ngrams across the sentence pair. Unlike most metrics, MAXSIM computes a similarity score between items, then find a maximum weight matching between the items such that each item in one sentence is mapped to at most one item in the other sentence. Evaluation on the WMT07, WMT08, and MT06 datasets show that MAXSIM achieves good correlations with human judgment.